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ABSTRACT 

Fourteen elementary teachers evaluated the quality 
and usefulness of 146 tapes produced by the Clinton Pilot Cassette 
Center, Minneapolis, Minn. Teachers' opinions, rather than tests on 
students, were the basis of the ratings. Judging from the ratings, 
most of the cassettes appear suitable for use with educationally 
disadvantaged youth and were in keeping with the intent of Title I 
guidelines. Also the general quality and content of the tapes were 
rated good. Cassette tapes recorded from radio broadcasts or 
purchased from other school districts were not as tjood for use with 
poor readers as those tapes purchased from commercial sources or 
produced at the cassette center- Commercial tapes were better 
technically, but center tapes were better for use with poor readers 
and also provided more "settling down" time for students at the start 
of the tapes. Since tv;o-thirds of the tapes were rated more suitable 
for intermediate than pri;iary grade children, more emphasis may need 
to be placed on making tapes for the lower grades. Cassette materials 
were judged by 70% of the raters to be more effective than other 
instructional materials for teaching the subject to disabled readers. 
A list of rated tapes, along with recommendations and evaluations, is 
i ncl u ded • (Author / JK) 
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Teachers' Rating Qt Cassettes Developed 
at the Clinton Pilov Cassette Center 
Clinton Elementary Schools 
Summer 1971 

Summary 

During the summer of 1971 fourteen Minneapolis Elementary 
School teachers participated in a workshop to evaluate the 
quality and appropiiateness of audio tapes produced by the 
Clinton Pilot Cassette Center. The goal of the Cassette See page 3 

Center is to improve instruction for children who learn 
better by simultaneously listening and viewing than they 
do by reading. The Center tries to reach this goal by pro- 
ducing films trips and audio tapes which help disabled readers 
succeed in school subjects despite their reading difficulties. 

The evaluation of the Cassette Center tapes was concerned 
with the quality and usefulness of the tapes themselves as 
viewed by teachers. During a two week summer workshop teachers 
rated 1^6 Cassette Center tapes which had been randomly drawn 
from the Cassette Center tape library. OnOy teachers' views See page K 
and opinions of the tapes were sought since students were not 
in school • No attempt was made to assess how efficiently or 
effectively the tapes were used during the regxilar school 
year nor was an attempt made to determine if students who 
used the tapes made significant improvement in basic skills 
subject areas. 

On the basis of the ratings made by the teachers, it appears 
that most of the cassette tapes were suitable for use with educa- See pages 
tionally disadvantaged youth and were in keeping with the intent 30 - 33 
and purpose of Title I ESEA guidelines. In addition, general 
quality and content of the tapes were rated good. 

Cassette tapes recorded from radio broadcasts or purchased 
from other school districts were not as good for use with poor 
readers as those tapes purchased from commercial sources or 
those made by the Cassette Center. Commercially produced 
tapes had better technical quality than Cassette Center pro- 
duced tapes, but Cassette Center tapes were rated better for 
use with poor readers. Cassette Center tapes were also much 
better than the other two types of cassette productions in 
the amount of "settling down" time provided students at the 
beginning of the tape. 

Four subject areas were eval.ua ted: Language arts, social 
studies, ioatheraatics and science. Most of the science tapes 
were found to be too difficult for use with K-6 grade level 
children. Mathematics was judged to be superior to the other 



See pages 
31 - 33 



See page 31 



subjects in the skills or concepts presented, but was found 
to be in need of improvement in the innerest level of the 
tapes • Language arts tapes needed more improvement than 
the other subjects in helping disabled readers learn the 
subject presented on the tape. 

Since two-thirds of the tapes were rated to be more 
suitable for intermediate than primary grade level childi*en, 
a question can be raised as to whether more emphasis needs 
to be placed on producing tapes for the primary grades. 

The cassette materials were Judged by 70^ of the ratings 
to be more effective than most other kinds of instructional 
\materials for teaching the subject to disabled readers. 



\ Appendix B lists those tapes which should be modified, 
those that should be scrapped, and those which should be 
kept without change or modification. Appendix C lists each 
taps separately along with each rater* s recommendation and 
evaluation. 

A number of recommendations are given. 



See page 31 



See page 33 



See page 31 



See page 32 



See page 33 
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Minneapolis Public Schools 

Teachers' Ratings of Cassettes Developed 
at the Clinton Pilot Cassette Center 
Clinton Elementary School 
Summer 1971 



The Clinton Pilot Cassette Center, located at Clinton Elementary 
School in Minneapolis, began operation on January 19, 1970 through a 
Title I ESEA initial grant of $9,825.^ A continuation grant of $36,383 
was funded for fiscal year 1971. The Center was planned to provide 
improved educ ational opportunities for target area children who were 
a year or more below grade level in reading achievement 'through extensive, 
effecth-.v and multiple use of cassette tapes, since poor readers have 
a grea'c i^al of difficulty reading printed material it was felt that 
listening-viewing activities would be a more appropriate mode of learning 
for poor readers. 

The major effort of the Center's staff (consisting of an audio- 
visual coordinator, resource teacher, and a clerk) since its inception 
has been to produce a library of cassette tapes suitable for small group 
instruction and individual student use. The teachers and the students 
at Clinton help to record tapes as it is felt that using many different 
voices prevents the tapes from becoming monotonous. Music is frequently 
used to introduce a lesson or to provide background. To date, over 60O 
tapes and related instructional materials in language arts, literature, 
mathematics, science, social studies, music and phj'sical education, have 
been purchased or produced for use with primary and intermediate age 
children. The tapes and related materials have been coded, boxed, and 
labeled for efficient circulation to the Clinton faculty and students. 
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From October 15, 1970 to May 28, 197I over 8,100 cassette tares 
were checked out of the Center's library by teachers and students for 
instructional or individual student use. Students may check out tapes 
and take them home or use them in school in the same manner they check 
out library books. Every afternoon, Monday through Thursday, students 
were allowed to take a cassette recorder and two tapes home. Only two 
tape recorders are available for overnight use in each classroom. In 
addition, each classroom had a listening center where children, using 
headphones, could listen to the tapes. 

A major focus of the project was to provide students with opportuni- 
ties to actual:^ achieve success over and over, thus building their self 
confidence. It was hoped that the net result would be better student 
attitudes toward school and self as well as improved school achievement. 

Objectives 

The major objectives of the Clinton Cassette Project were: 

^' I^o^r^^°? ^ i'^''!'^ °^ cassette tapes for class, small 
^oup, and individual listening which will be suitable 
for use with educationally disadvantaged youth. 

2. To d.e cassette tapes which are suitable for the -.any 

filmstrips now available for disabled readers. ^ 

^' skillf r^fS'r\'"/'^'^°P^"8 competency in listening 
Skills related to basic skills achievement. 

It-T^^^ students acquire competency in learning, and gain 
skill in the techniques of inquiiy and critical examination. 

^* IrrZ, ^! ^ resource agency for persons needing help in the 
lefiniS a?d! " ^" instrucSonal a^d 

^' ll^llll '^^J'"" *heir own lesson (interviews, class 

reports, book reviews), * v.-i.ci.oo 

^* recordLfl^^^.^'T presented via cassette 

recorders in their classrooms. 

^' optional instructional methods to the classroom 

not otherwise possible; children would learn by listening 
as well as by reading. ^ ^ * 



Program Description 



The goal of the Cassette Pilot Center was to develop a tape library 
that would help improve instruction for children who learn better by 
simultaneous listening and viewing theui by reading so that disabled 
readers, would not fail in other basic skills areas. The project served 
primarily kO^ of the 376 children at Clinton Elementary School who 
achieved at the twenty-fifth percentile or lower on standard achievement 
tests in reading and mathematics. 

The Clinton Cassette materials and tapes wer^ produced by several 
methods: First, tapes were produced in the Center's recording studio with 
the help of teachers nnd sometimes students. Secondly, the Center pur- 
chased tapes from various commercial enterprises and modified them when 
needed to suit local purposes. Thirdly, radio programs that appeared 
to be suitable for the Center's purposes were either recorded or purchased 
by the Center and modified when needed. Finally, in several instances 
the Center bought tapes which were produced by other school district?^. 

Cassette materials including filmstrips, tapes, transparencies, slides 
and mimeographed materials were used for instruction in class, small 
groups and individual listening. Many of the recorded lessons were geared 
to students' textbooks in science, social studies, language arts and math. 
Thus students who were poor readers and who easily became frustrated and 
discouraged when reading a lesson, could listen to the lesson being read 
to them on tape and follow along in the textbook. Poems, short stories, 
readings in literature and even practice in multiplication tables were 
available for student use and could be taken home for overnight listening 
and study. 




The Center's staff prepared and developed cassette materials, helped 
teachers >M students use the materials and equipment effectively, and 
organized and maintained the cassette library so that teachers and stu- 
dents could easily retrieve appropriate tape and related materials. 
Proquentl,, supplementary materials such as fil^strips, mimeographed 
worksheets etc., were included in the box containing the cassette tapes. 

Evaluation 

one of the Cassette Center's major purposes was to develop a library 
Of cassette tape lessons which would be suitable for use with Title I, 
ESEA students. The evaluation of the Center's operations «>s restricted 
to an analysis of the quality and usefulness of the cassette tapes themselves 
as Viewed by teachers. The study was c«,ducted during a Wo week s^er 
workshop by 1- teachers who rated 1-6 Cassette Center tapes which had 
been randomly drawn f^m the Cassette Center tape library. On^ teacher 
Views and opinions of the tapes were sought since students were not in 
school. Thus, no attempt was made to assess hp. efficiently or effectively 
the tapes were used during the re^lar school year and no attempt was „.de 
to determine if students who used the tapes made significant i^ove^nt 
in basic skiUs subject areas. The evaluation has attempted to provide 
answers to the following questions: 

1. Which cassettes appeared to be well sint.rt » 
be-:tatens— T^tVi^^^^^^ 

ki^dTc^-ellh^SdT - - ^ -d 

u"^ t?cai:ntt%snr«r\^^^^^^^^^^^ ™ 

cassettes measiire ud to thP tnn<.c v^«,, v7 ^ ^ 
broadcasts and thosp fLl^ ^ recorded from radio 

distrJc?s? ^ ^^^"^ purchased from other school 

ofprj^tation, content interest, etc.? 



In order to achieve the above objectives it was deterndned that a 
representative sampling of each of the major subject areas (language arts, 
mathematics, social studies and science) cassette tapes be taken and that 
the tapes be rated or evaluated by t knowledgeable group of teachers who 
had previous instructional experience with youngsters for whom the cassettes 
were intended. 

Fourteen teachers were selected to conduct the cassette tape evaluation. 
Each principal from the ih Title I South Pyramid elementary schools which 
included two parochial schools, chose one teacher to represent his school. 
(The South Pyramid is a geographicalQjr unified group of schools organized 
into a unit for decentralized planning and administration). Of the ik 
teachers selected, all had four years teaching experience or more; four 
were primary teachers, eight were intermediate teachers and two were 
librarians. 

One-hxindred forty six cassettes were randomly selected for evaluation 
from a list of U8l tapes which the Center had on file in its library as of 
May 1, 1971. The sample accounted for 30^ of the tapes in the four major 
subject areas. Table 1 indicates the breakdown for the sample and the 
number of tapes that were evaluated by subject area. 

Table 1 

Summary of Cassette Sampling Procedures by Subject Area, 
Number in Each Subject Area, Number Selected for 
Evaluation and Percent Sampled 
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Number 


Percent 


Subject Area 


Number 


Selected 


Sampled 


Language Arts and Literature 


20k . 


59 


29 


Social Studies 


175 


5U 


31 


Mathematics 


73 


22 


30 


Science 


29 


11 


38 


Total 


U8l 


Ike 


30 




5 


14 





Beginning June 21, 1971 at the Clinton Elementary School, the ik 
teacher raters were given a brief orientation on how to evaluate the tapes. 
A cassette rating sheet (see Appendix A for Cassette Evaluation by Teachers), 
developed by the Research Deportment in cooperation with the Clinton 
Cassette staff, was provided and each teacher was required to fill out 
one rating sheet for each tape evaluated. TypicalOjr, each tape was rated 
by four teachers although several tapes received fewer than four and 
several received more than four evaluations (see Table 2 for subject break- 
down of number of ratings and Appendix C for the number of ratings for each 
individual tape). 

Each day teachers were given rating sheets on which cassette numbers 
had been entered. The cassette numbers had been randomly assigned to 
assure that no bias entered into the order m which the teachers rated 
the tapes , thus teachers worked independently of one another. As an 
example, one rater might do a certain social studies cassette, then a 
science tape, then a math tape. Another teacher might be doing a social 
studies tape, a math tape and then a social studies tape. The teacher 
raters had no knowledge of the source of production of the tape. 

The Ik raters worked ten days on the project. One-hundred forty six 

tapes were evaluated with 55U teacher ratings made about the tapes. 

Generally headsets were used by the teachers to listen to the tapes as 

is the case when individual children listen to the tapes in school. 

Ninety-two percent of the ratings were made by teachers using headsets. 

No instances were reported by the teachers which indicated trouble using 

the cassette recorders or headsets. 

The section which follows presents the results of the teacher ratings. 
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Results 

The results are presented in this section under three general headings. 

The first section reports the results by subject areas: language arts, 
social studies, mathematics and science. Individual cassette evaluations 
and rater comments are recorded in Appendix C. 

Section two describes the results by production source; i.e., whether 
the tapes were produced by the cassette center, by a commercial enterprise 
or recorded from radio or purchased from some other school district. 

Section three reports the results according to whether the tapes were 
suitable for primary or intermediate grade level use. 

Cassette Tapes Analyzed by Subject Area 

Table 2 shows the number of tapes evaluated by subject area and the 
number of individual ratings made for each subject area. In all, lU6 
tapes were evaluated with a total of 55U ratings made. The average number 
of ratings for each tape was 3.9. 

Table 2 

Number of Tapes Evaluated, Total 
Number of Ratings and Mean 
Ratings by Subject Matter 



Subject 


Number 
Rated 


Total 
Ratings 


Mean Ratings 
For Each Tape 


Language Arts 


59 


223 


3.8 


Social Studies 


5U 


211 


3.9 


Mathematics 


22 


81 


3.9 


Science 


11 


39 


3.9 


Total 


IU6 


55U 


3.9 



Table 3, page 9 presents the results by subject area when the teacher 
evaluators were asked to give a very good, good, fair or poor rating to each tape 
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the following questions: 



1. Was the voice clear? 

2. Was the voice pleasant? 

3. Was the sound good? 

U. Was the pacing of the materials presented appro- 
priate for the children? 

5. Was the presentation of optimum length; neither 
too long nor too short? 

6. Was sufficient "settling down" time provided at the 
beginning of the tape? 

7. Was the content relevant for children? 

8. Was the content interesting? 

9. Were the related materials appropriate for the 
Cassette presentation? 



Eight of the nine tape quality items had an overall rating of good 
or very good in &Sfi> of the cases. The range for the eight criteria items 
was from 893t to 97% with good or very good ratings when all subjects were 
considered. 

Only settling down time received less than 89i> good or very good ratings. 
It was accorded a good or very good rating 7^^ of the time and received 17% 
poor ratings. Language arts settling time tapes received a poor rating 22% 
of the time by the e valuators while 67% of the ratings were good or very 
good on settling time. Social studies had 17% poor ratings, science 10%, 
and mathematics 6%. 

Mathematics had a content interest rating of good or better 6h% of 
the time, but had 17% poor or fair ratings. 

Table k on page U reports the results by subject matter areas when 
the tapes were evaluated for their effectiveness in helping disabled readers 
learn the subject presented. Specifically, raters were asked to give a tape 
a very effective, effective, of limited effectiveness or worthless rating 
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Table 3 



Frequency and Percent Distributions for All Subject Areas 
And By Language Arts, Social Studies, Mathematics and 
Science When the Cassette Tapes Were Rated on 
Nine Tape Quality Items^ 







All 


Language 


Social 












Subjects 


Arts 


Studies 


Mathematics 


Science 


T)s 't*^ nry 


N 




N 




N 




N 




N 




Voice Clarity 






















Very Good 


2o7 


52 


120 




112 


53 


oil 

3*+ 


liO 

H2 


21 


5** 


Good 




**5 


91 


Hi 


nil 
94 


li c 

**5 


*+5 




10 


HO 


Fair 


10 






q 
P 


q 
P 




c 




V 


0 


Poor 


1 


0 


1 


0 


0 


0 


0 


0 


0 


0 


Total 




100 


223 


100 


211 


100 


81 


100 


39 


100 
























Very Good 


0^*7 
dOf 


HO 


Hi 


51 


oft 


li7 




HH 




pi 


Good 




lit; 




HU 






HH 


pH 


10 


Ul 

Hi 


Fair 




D 




n 
y 


1^ 


D 


1 


1 


J 


P 


Poor 


2 


0 


1 


0 


1 


0 


0 


0 


0 


0 


lova J. 


553 


99 


223 


100 


210 


100 


81 


99 


39 


100 


Sound Quality 
















Ufl 

*tO 




po 


Very Good 


HO 


lU^ 


Hp 




HO 


^y 


C.C. 


Good 


2U7 




Q7 


uu 


07 






UU 


17 


UU 


Fair 




J 


J- J 


6 


11 


J 


6 


7 


0 


0 


Poor 


12 


2 


11 


J 


1 


0 


0 


0 


0 


0 


Total 




lUU 


223 


lUvj 


Oil 

211 


99 


01 


99 


39 


1 Tin 
lUU 


Appropriateness of 






















Pacing 














36 








Verv Good 


250 


**5 


99 


1.1. 

HH 


95 


1. c 

**5 


1.1. 
UU 


20 


51 


Good 


252 


Ub 


105 


l.rr 

U7 


93 


i.ii 

HH 


37 


ii^ 
Uo 


17 


HH 


Fair 


uo 


7 


12 


5 


19 


9 


7 


9 


2 


5 


Poor 


11 


2 


7 


3 


3 


1 


1 


1 


n 
U 


n 
U 


Total 




100 






210 


QQ 
yy 


81 


100 


'^Q 


100 


Length of Presenta- 






















uxon 






















Very Good 


2U3 




103 


U7 


92 


uu 


33 


Ul 


15 


38 


Good 


265 


U8 


105 


US 


95 


U6 


U2 


52 


23 


59 


Fair 


35 


6 


11 


5 


19 


9 


U 


5 


1 


3 


Poor 


6 


1 


2 


9 


2 


1 


2 


2 


0 


0 


Total 


5U9 


99 


i?21 


99 


208 


100 


81 


100 


39 


100 


Settling Time 








3U 














Very Good 


2U 


39 


75 


81 


. 39 


uo 


U9 


15 


38 


Good 


191 


35 


73 


33 


69 


33 


30 


37 


19 


U8 


Fair 


5U 


10 


2U 


11 


23 


11 


6 


7 


1 


3 


Poor 


92 


17 


U8 


22 


35 


17 


5 


6 


u 


10 


Total 


5U8 


101 


220 


100 


208 


100 


81 


99 


39 


99 
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Table 3 (continued) 



Rating 


All 
Subjects 

N i 


Language 

Arts 
N f, 


Socinl 
Studies 

N i 


Mathematics 
N ^ 


Science 
N % 


Content Relevance 
Very Good 
Good 
Fair 
Poor 

Total 


296 53 
218 39 
3h 6 
6 1 

55U 99 


108 U8 
95 U3 
17 8 
3 1 

223 100 


12h 59 
71 3^ 
13 6 
3 1 

211 100 


U8 59 
31 38 
2 2 
0 0 

81 99 


16 Ul 
21 5U 
2 5 
0 0 

39 100 


Content Interest 
Very Good 
(iooa 
Fair 
Poor 

Total 


257 hi 
2U3 hh 
hh 8 
5 1 
5U9 100 


98 U5 
101 hb 
18 8 
2 :. 

219 100 


119 56 
78 37 
13 6 
1 0 

211 99 


2h 30 
U3 5^ 
11 Xh 
2 3 
80 101 


16 Ul 
21 51* 
2 5 
0 0 

39 100 


Appropriateness of 
Accompanying Materials 

Very Good 

Good 

Fair 

Poor 

Total 


205 U9 
166 Uo 
32 8 
ih 3 

U17 100 


73 hi 
68 U3 
8 5 
8 5 

157 100 


82 5^ 
U9 32 
17 11 
3 2 

151 99 


37 h6 
37 U6 
U 5 
3 h 

81 101 


13 h6 

12 U3 
3 11 
0 0 

28 100 



N varies slightly in some instances because the raters failed to respond to an 
evaluation item, N is considerably smaller for appropriateness of accompanying j 
materials than other rating categories because in many instances there were no 
accompanying materials for the cassette tapes. In some instances percent totals 
do not equal 100% because of rounding, 

i 
i 
( 
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to the following question: In your opinion, how effective would this 
lesson be to help severely disabled readers learn the subject presented? 



Table k 

Frequency and Percent Distributions for All Subject Areas 
and by Language Arts, Social Studies, Mathematics and 
Science When the Cassette Tapes Were Rated for Their 
Effectiveness in Helping Severely Disabled Readers 
Learn the Subject Presented 



Ratings 


All 
Subjects 
N t 


Language 

Arts 
N t 


Social 
Studies 

N IL 


Mathematics 

N <i. 


Science 
N i 


Very Effective 

Effective 

Of Limited Effec- 
tiveness 


2iy 
265 
55 


UO 
U8 
10 


llU 
2k 


33 
51 
11 


95 
96 
16 


U5 
U6 
8 


31 
38 
12 


38 
15 


19 k9 
17 Uk 
3 8 


Worthless 


lU 


2 


11 


5 


3 


1 


0 


0 


0 0 


Total* 


553 


100 


223 


100 


210 


100 


81 


100 


39 101 



In some instances percentage totals do not equal 100% because of rounding. 

When all subjects were considered, m of the tape ratings were Judged as 
being effective or very effective in helping disabled readers learn the subject 
matter. Science tapes had a 93% effective or very effective rating, social 
studies 91%, mathematics 85"^, and language arts 6k%, Two percent of the ratings 
considered the tapes to be worthless as far as helping disabled readers learn 
subject matter. 

Table 5 reports the results when the cassette tapes were compared with 
other instructional materials for teaching the subject to disabled readers. 
Raters were asked .he following question: Compared with other instructional 
materials you have used, how would you rate this cassette for teaching the sub- 
ject to disabled readers? The raters then checked one of the following responses: 
(l) The most effective material I know of for this subject, (2) More effective 
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than most materials on this subject, (3) About as effective as most other 



materials on this subject, (k) Less effective than most materials on this 
subject. 

Table 5 

Frequency and Percent Distributions For All Subject Areas 
And By Language Arts, Social Studies, Mathematics and 
Science When the Cassette Tapes Were Compared With 
Other Instructional Materials for Teaching 
The Subject to Disabled Readers 



All 
Subjects 



Language 
Arts 

N 



Social 
Studies 



N 



Mathematics Science 
U i }i % 



Most effective material 
I know of for this 
subject 


95 


17 


29 


13 


kk 


21 


12 


15 


10 


26 


More effective than 
most materials on 
this subject 


296 


53 


117 


52 


116 


55 


k2 


52 


21 


5k 


About as effective as 
most other materials 
on this subject 




22 


63 


28 


36 


17 


20 


25 


5 


13 


Less effective than 
most materials on 
this subject 


39 


7 


Ik 


6 


15 


7 


7 


9 


3 


8 


Total^ 


55U 


99 


223 


99 


211 


100 


81 


101 


39 


101 



^In some instances percentage totals do not equal 100^ because of rounding- 

Seventy percent of the ratings reportea that the cassette materials 
were more effective than most other instructional materials for teaching 
the various subjects to disabled readers. Seven percent indicated that 
the materials were less effective than other materials on the subject. In 
less than 10^ of the cases did teachers feel that the cassette materials 
were less effective than most materials. Over 90^ of the ratings judged 
the cassette materials to be at least as good as most other materials and 70?t 
of the ratings indicated that the cassettes were either more effective or 
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the most effective materials that the raters had seen on the topics 
covered by the tapes. 

Table 6 reports the findings when the lU evaluators were asked to 
rate the appropriateness of the materials for poor, average, or better 
than average readers, A "poor" reader was defined as a child who is 
one or more year3 below grade level. 



Table 6 

Frequency and Percent Distributions for All Subject Areas 
And by Language Arts, Social Studies, Mathematics and 
Science When the Cassette Tapes Were Rated for 
Good, Average or Poor Readers 



Rating 



All 
Subjects 
N % 



Language 
Arts 

N <i 



Social 
Studies 



N 



Mathematics Science 



I 
I 



More appropriate for 
poor readers than for 
average or better than 
average readers? (A 
"poor" reader is one or 
more years below grade 
level) , 

More appropriate for 
average or better than 
average readers than 
for poor readers 

Appropriate for good, 
poor and average 
readers alike 

Not appropriate for 
good, poor or average 
readers 

Total^ 



191 



35 77 



35 



76 36 



25 32 13 33 



71 



277 



39 18 



22 



11 



51 203 '+7 108 52 UU 



56 



22 



5'+6 100 220 100 209 



10 



56 



100 78 100 39 99 



In some instances percentage totals do not equal 100^6 because of rounding. 

Table 6 indicates that 3% of the ratings for all subject areas 
judged the tapes more appropriate for poor readers than for average or better 
than average readers. Fifty-one percent of the tape ratings were reported to be 



appropriate for good, poor and average readers alike. Thus, 86^t of the 
tape ratings indicated the cassettes to be suitable for poor readers. 
Eighty-two percent of the language arts ratings indicated that the tapes 
were suitable for use with poor readers, social studies had 88^6, mathe- 
matics 88^, and science 89^ favorable ratings. Only one. percent of the 
ratings reported that the cassettes were not appropriate for good, poor 
or average readers. 

Table 7 reports the resxats when the tape?? were judged to see how 
important the skill, concept or information contained on the tape was 
for instruction. 



Table 7 

Frequency and Percent Distributions for All Subject Areas 
And by Language Arts, Social Studies, Mathematics and 
Sciejnce When the Cassette Tapes Were Rated To 
Determine How Important the Skill, Concept 
or Information Presented for Instruction 
Was in the Subject Area 



RatinR 


All 
Subjects 
N t 


Language 
Arts 
N ^ 


Social 
Studies 
N H, 


Mathematics 

N <s; 


Science 
N <i 


Crucial, can't do with- 
out it 


79 


lU 


28 


13 


23 


11 


25 


31 


3 


8 


Very important 


272 


U9 


109 


U9 


99 


U7 


kk 


5U 


20 


51 


Moderately important 


192 


35 


81 


36 


8U 


Uo 


22 


15 


15 


38 


Not very important 


7 


1 


k 


2 


2 


1 


0 


0 


1 


3 


Trivial 


k 


1 


1 


0 


3 


1 


0 


0 


0 


0 


Total* 


55^ 


100 


223 


100 


211 


100 


81 


100 


39 


100 



ERIC 



In some instances percentage totals do not equal lOQ^ because of rounding. 

Sixty-three percent of the ratings judged the tapes to be very important 
or crucial in regard to the skill, concept or information presented. Individual 
subject breakdowns were: Language arts 62%, social studies 58^t, mathematics 85%, 
and science 59^. One percent or less of the tape ratings judged the tapes as not 

23 

lU 



very important or trivial. 



j Table 8 reports the results when the teacher raters were asked the 

following question: What is your recommendation for this cassette pre- 
j sentation? (l) Keep it as is, (2) scrap it or (3) keep it, but make the 

j- following changes. Readers wanting to know what changes were recommended 

should see Appendix C which has each tape listed separately. 

I 

Table 8 

Freiuency and Percent Distributions for All Subject Areas 
And by Language Arts, Social Studies, Mathematics and 
Science When the Cassette Tapes Were Rated for 
Keeping Tape As Is, Scrapping 
It, or Modifying It 



All Language Social 



Subjects Arts Studies Mathematics Science 

Rating - ^ 



Keep it as is 


3^ 


70 


15U 


70 


152 


/» 
73 


50 


10 

63 


n 
28 


7b 
72 


Scrap it 


19 


k 


10 


5 


6 


3 


3 


k 


0 


0 


Keep, but modifV 


1U5 


26 


56 


?5 


51 


2k 


27 




11 


28 


Total* 


5U8 


100 


220 


100 


209 


100 


80 


101 


39 


100 



In some instances percentage totals do not equal 100% because of rounding 

Seventy percent of the ratings recommended that the cassette tapes 
be maintained as they are. Mathematics had a keep rating of 63% which 
was the lowest of the subject areas. Social studies had the best rating 
with 73% of the raters recommending that the tapes should be kept in their 
present condition. 

Four percent of the tape ratings had a scrap rating, with science 
having none and language arts (five percent) the highest. 

The keep-but modify recommendation was made in 26% of the ratings, 
with science having none of these recommendations and mathematics having 
the largest {^H). 



15 



Table 9 indicates the number and percent of cassette tapes by sub- 
ject area that received a unanimous recommendation on each tape by those 
rating it that it should be kept in its present condition or scrt^pped. 
Also listed are those tapes having one or more modify recommendations* 

Table 9 

Number and Percent of Cassette Tapes by Subject Area That 
Received a Unanimous Recommendation on Each Individual 
Tape by Those Rating That It Should Be Kept in 
Its Present Condition or Scrapped and Also 
Those Tapes Having One Or More 
Modify Recommendations 



All Language Social 

Subjects Arts Studies Mathematics Science 



Rating 


N 




N 




N 




N 




N 


i 


Keep 




31 


19 


32 


19 


35 


5 


23 


2 


18 


Scrap 


2 


1 


0 


0 


2 


k 


0 


0 


0 


0 


ModifV 


99 


68 


ko 


68 


33 


61 


17 


77 


9 


82 


a 

Total 


1U6 


100 


59 


100 


5U 


100 


22 


100 


11 


100 



^In some instances percentage totals do not equal 100^ because of rounding. 

Table 9 indicates that U5 of tho lh6 tapes rated {31%) received a unanimous 
keep rating; that is^ if a tape had four ratings all four ratings recommended 
keeping the tape without any modification whatsoever. Two tapes had a scrap 
recommendation (both in social studies) and 99 (68^) of the tapes had one 
or more recommendations recommending that the tape should be modified in 
some manner. 

Language arts and science had the highest percentage of keep ratings 
with 32^ and 35% respectively. Science and mathematics had the highest 
percentage of modify recommendations with science having 82% and mathema- 
tics n%. 
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Cassette Tapes Analyzed by Productiori Source 

The Cassette Center produces its own tapes, b\iys tapes from commercial 
enterprises and other school districts, as well as recording live radio 
broadcasts. For the purposes of this report the tapes have been placed 
into three groups and have been analyzed accordingly. The following 
describes the grouping procedure and abbreviations used in subsequent 
tables: 

1. Local means that the tapes were produced by the Cassette Center, 

2. Commercial means that the tapes were produced by a commercial 
enterprise which sells educational tapes for a profit. 

3. Radio and other schools means that the tapes were recorded from 
a live radio broadcast or that the tape was produced and pro- 
curred from another school district. 

Table 10 presents a breakdown of the various levels of production 
by subject area showing the number of tapes that were produced locally, 
commercially and by radio and other schools. 

Table 10 

Number and Percent of Cassettes by Production 
Source and Subject Matter 

Radio and 

Local Commercial Other Schools 

Subject N 



Language Arts 


20 


33 


35 


U8 


U 


33 


Mathematics 




20 


8 


11 


2 


17 


Science 


k 


7 


7 


10 


0 


0 


Social Studies 


25 


Ul 


23 


32 


6 


50 


Total* 


61 


101 


73 


101 


12 


100 



Percentage totals may not equal 100^ because of roxmding. 
Table 10 shows that 6l tapes were produced locally, 73 commercially, 
and 12 by radio and other schools. Local and commercial productions had 
about the same percentage of tapes by subject matter. However, radio and 
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school productions had 50* of its tapes in social studies while not 
having any productions in science. 

Table 11 shows the quality of tapes by production level using the 
nine criteria items given on page 8 . 



Table 11 



Frequency and Percent Distributions by 
Cassette Production Soxirce on Nine 
Tape Quality Criteria Itemsa 



Ratin, 



Local 




Radio and 
Commercial Other 
N % H 



Voice Clarity 
Very Good 
Good 
Fair 
Poor 

Total 



283 
2U6 
18 
1 



52 
U6 

3 
0 



9h 
115 
12 
1 



U2 
52 
5 
0 



13U 
73 
3 
0 



6U 
35 
1 
0 



55 
58 
3 
0 



Pleasantness of V oice 
Very Good 
Good 
Fair 
Poor 

Total 

Sound Quality 
Very Good 
Good 
Fair 
Poor 

Total 

Appropriateness of Pacing 
Very Good 
Good 
Pair 
Poor 

Total 



263 
2U8 
3h 
2 



U8 
U5 
6 
0 



5U7 99 



86 


39 


126 


60 


111 


50 


79 


38 


23 


10 


k 


2 


1 


0 


1 


0 


221 


99 


210 


100 



51 

58 
7 
0 

116 



262 


U8 


93 


k2 


122 




h5 


108 


h9 


7U 


30 


5 


17 


8 


6 


12 


2 


k 


2 


8 


5U8 


100 


222 


101 


210 



2U8 
2U9 
39 
11 



5U7 



U5 


92 


Ul 


112 


U6 


106 


U8 


86 


7 


20 


9 


7 


2 


k 


2 


5 


100 


222 


100 


210 



53 
Ui 

3 
2 



57 
12 
2 



Total 
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5U3 99 

27 



86 


39 


109 


52 


111 


51 


92 


kk 


16 


7 


9 


h 


5 


2 


0 


0 


218 


99 


210 


100 



U5 
59 
10 
1 

115 



U7 
50 
3 
0 




hk 
50 
6 
0 



58 


^7 


Ui 


35 


62 


53 


3 


7 


6 


k 


0 


0 


100 


116 


100 



38 
50 
10 
2 




39 
51 
9 
1 

100 



Table 11 (continued) 



Rating 



Settling Time 
Very Good 
Good 
Fair 
Poor 

Total 



Total Local Commercial 

N ^ N * N «i 



Radio and 

Other 
_N ^ 



210 
188 
53 
91 

5U2 



39 
35 
10 
17 
101 



129 
79 
8 

3 

219 



59 
36 
U 
1 



58 
78 
26 
U6 

100 208 



28 

38 

13 

22 



23 
31 
19 
U2 



101 115 



20 
27 
17 
37 

101 



Content Relevance 
Very Good 
Good 
Fair 
Poor 



Total 

Content Interest 



29t» 
21U 
3U 
6 

5U8 



5U 

39 
6 
1 

100 



117 

88 
lU 
3 

222 



53 


115 


55 


62 


Uo 


81 


39 


U5 


6 


13 


6 


7 


1 


1 


0 


2 


100 


210 


100 


116 


U7 


100 


U9 


52 


U3 


89 


U3 


5U 


8 


16 


8 


10 


2 


1 


0 


0 


100 


206 


100 


116 



53 
39 
6 
2 

100 



Very Good 
Good 
Fair 
Poor 

Total 



255 

239 
kk 

5 

5U3 



U7 
kk 
8 
1 



103 
96 
18 
k 



100 221 



U5 
U7 
9 
0 

101 



Appropriateness of 
Accompanying Materials 

Very Good 
Good 
Fair 
Poor 

Total 



203 
I6U 
32 
Ik 



k9 
ko 
8 
3 



110 

88 

15 
2 



51 
Ul 

7 
1 



71 
61 
10 
7 



U13 100 215 100 1U9 



U8 
Ul 
7 
5 

101 



22 
15 
7 
5 

U9 



U5 
31 
lU 
10 

100 



N varies slightly" in some instances because the raters failed to respond to an 

J°nsiderab2y smaller for appropriateness of accompanying 
materials than other rating categories because in many instances there were no 
accompanying "^rials for the Cassette tapes. In some instances oercent toSals 
do not equal 100^ because of rounding. 
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If one looks at the ratings of good or better for the nine quality 
criteria items, little or no difference can be distinguished among the 
three production sources. With the exception of "settling time" and 
"appropriateness of related materials" all three production sources had 
a good or very good rating 88^ or more of the time with an overall rating 
equal to or exceeding 91% • 

Commercial productions had a higher percentage (lO-20lt) of very good 
ratings than either local or radio productions on the first five criteria 
items. Hovever the commercial production ratings on settling time had 
only 66^ good or very good ratings compared to local productions which 
had 95% of the ratings at or above the good rating level. Twenty- two 
percent of the commercial production ratings on "settling time" were 
Judged to be poor. Radio and other school production had only kTio good or 
better ratings while having 37% poor ratings. 

Radio and other school productions appear to be in need of improve- 
ment in the related materials depcLrtment. Twenty- four percent of the 
ratings for related materials were fair or poor. 

Table 12 reports the findings when the tapes were evaluated by pro- 
duction source to determine the source effect on helping disabled readers 
learn the subject presented, i.e. were local productions better than 
commercial and radio productions, were they the same or were commercial 
productions better for disabled readers than either of the other two 
productions? Table 12 indicates that apparently production source had 
little relationship to whether or not the cassettes were Judged to be 
effective in helping disabled readers learn subject matter. It should 
be noted, however, that radio and other school productions were reported 
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to be somewhat less effective than the other two production sources with l6% 



of the ratings indicating limited effectiveness. 



Table 12 

Frequency and Percent Distributions for All Production 
Sources and by Local, Commercial, and Radio or Other 
Schools When the Cassette Tapes Were Rated for Their 
Effectiveness in Helping Severely Disabled Readers 
Learn the Subject Presented 







All 










Radio 


and 




Productions 


Local 


Commercial 


Other 


Bating 


N 




N 




N 


i> 


N 




Very Effective 


216 


39 


90 


Ul 


86 


Ul 


UO 


35 


Effective 


262 


U8 


112 


50 


96 


U6 


5U 


U7 


Of Limited Effectiveness 


55 


10 


18 


8 


19 


9 


18 


16 


Worthless 


lU 


3 


2 


1 


9 


h 


3 


3 


Total^ 


51*7 


100 


222 


100 


210 


100 


115 


101 



*In some instances percentage totals do not equal 100% because of rounding. 



Table 13 reports the findings when the ratings were analyzed by source 
of cassette production to determine the cassette tapes effectiveness when 
compared to other instructional materials • The table indicates that the 
materials were about as effective regardless of production source; i.e., 70% 
of all productions were more effective or better than other materials the teacher 
raters had used for helping disabled readers. Local productions had 73% more 
effective or most effective ratings, commercial 73%, radio and other sources 6U%. 
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Table 13 

^^n^II7T^"^^'^'"* Distributions for All Production Sources 
And by Local Conunercial, and Radio or Other Schools Whe^ 
The Cassette Tapes Were Compared With Other InstructiorlJ 
Materials for Their Effectiveness in Teaching 
Disabled Readers 




Ratin 

Most effective material 
I know for this subject 

More effective than most 
material on this subject 

About as effective as 
most other materials on 
this subject 

Less effective than 
most materials on this 
subject 

Total^ 



11 



16 



12 



Percentage totals may not equal locyf, due to rounding. 
Table Ik reports the results when the tapes were analyzed by production 
source to determine how effective the various sources arc with poor, average, 
and good readers (A "poor reader" is a child who is one or more years below ^ 
grade level). 
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Table lU 

^ITZ"^ ^r*'^"* Distributions for All Production Sources 
r^t^..^^^^* Commercial and Radio or Other Schools Shen^^e 
Cassette Tapes Were Rated for Their Effectiveness in HeL^l!^ 

Poor, Average and Good Readers 



Ratir 

More appropriate for poor 
readers than for average 
or better than average 
readers 

More appropriate for aver- 
age or better than average 
readers than for poor readers 

Appropriate for good, poor 
and average readers alike 

Not appropriate for good, 
poor, or average readers 
Total^ 



All 

Productions Local 
190 35 100 



69 13 13 



Commercial 
N 



Radio and 
Other 

N 



he 


60 


29 


30 


26 


6 


32 


16 


2h 


21 


he 


113 


55 


61 


53 


3 


1 


0 


0 


0 


101 


206 


ICQ 


115 


100 



Thirty-five percent of the ratings for all productions indicated 
that the cassettes were more appropriate for poor readers than for 
average or better than average readers. Local productions vere considerably 
better {k6%) than either commercial (295^) or radio and other schools (26^) 
on this category. In addition, 32% of the commercial, and 21^ 
of radio and other schools were judged to be more appropriate for average 
or better than average readers than for poor readers. Since only 13% 
of the local productions received this rating it appears that local pro- 
ductions do a better job of preparing tapes for poor readers. Little or 
no difference existed among production sources for the other two ratings. 

Table 15 reports the findings when the tapes were analyzed by production 
source to determine how important the skill, concept or information contained 
on the cassettes were for instructional purposes. 

Table 15 

Frequency and Percent Distributions For All Production Sources 
and By Local, Commercial, and Radio or Other Schools When The 
Cassette Tapes Were Analyzed to See How Important the Skill, 
Concept or Information Presented on the Tape was 
For Instructional Purposes 

All Radio and 

Productions Local Commercial Other 



Rating N j N ^ N % N % 



Crucial, can't do 
without 


79 


-itr- 


30 




37 


18 


12 


10 


Very important 


268 




111 


50 


102 


U9 


55 


hi 


Moderately important 


190 


35 


76 


3U 


68 


32 


U6 


ko 


Not very important 


7 


1 


3 


0 


2 


1 


2 


2 


Trivial 


k 


1 


2 


1 


1 


0 


1 


1 


Total* 


5U8 


100 


222 


99 


210 


100 


116 


100 



^Percentage totals may not equal 100^ due to rounding. 



32 

23 



Table 15 indicates that no major differences existed among the 
three different production sources. The material contained in the 
cassettes was judged to be moderately important or better by 98^t of the 
ratings. Very important and crucial ratings exceeded 57* in all levels 
of production with commercial (67%) the best and radio and other school 
productions the poorest (57^t). 

Table 16 presents the keep, scrap or modifV recommendations when 
viewed from e production level; that is, was there a greater percentage 
of tapes for a certain production source that had a keep without modifica- 
tion rating than the other productions? Did one production source have 
more of its tapes with scrap recommendations? 



Table 16 



Frequency and Percent Distributions for all Productions and by 
Local, Commercial, Radio and Other Schools Productions When the 



Rating 


All 
Productions 
N i 


Local 
N <i, 


Commercial 
N * 


Radio and 
Other 


Keep it as is 


380 


70 


157 


72 


IU9 


71 


N ^ 

7k 65 


Scrap it 


19 


k 


7 


3 


7 


3 


5 k 


Keep but modiiy 


IU3 


26 


55 


25 


53 


25 


35 31 


Total* 
a 


5U2 


100 


219 


100 


209 


99 


Ilk 100 



in some instances percentage totals do not equal lOO^t because of rounding. 
The mean keep rating for all productions was 70% with the Cassette 
Center productions having the best rating (72%) and radio and other schools 
((>yf>) the poorest. 

Pour percent of the ratings recommended that the tapes be scrapped with 
little variation among the three production sources. 

Twenty-six percent of the tapes had a modify recommendation with local 
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and co««,rcial prcxiuctlons having a 2% ^ity r,co«nendatio„. Radio and 
other schools had 31« of the ratings re=o.u«nding change or .odifioation. 

It appears that little difference existed between Cassette Center 
locall, produced tapes and co«erciall, produced tapes when viewed fro„ a 
keep, scrap or ^.iifjr rating. Badio and other school production had a 
higher modification rate and would seem to be somewhat poorer than the 
other two production sources when rated on a keep, scrap, or modify basis. 

Cassette Tapes Analyzed Bv Pr^n,^ .... intermediate r.^.. 

Table 17 reports the results when the tapes were Judged for their 
suitability for use with pri«>ry (K-3) or intermediate (^-6) grade children 
by sub^Ject area. 



Table 17 

^Anf J''^^"'^ Distributions For All Subject Areas 

And by Language Arts, Social Studies, MathemaJics and 
Science When the Cassette Tapes Were Rated^oJ use 
With Primary and Intermediate Children 

Language gbcia" 
Subjects Arts Studies 

N * N * N 



Mathematics 
N 



Science 
N 



37 


2k 


13 


15 


22 


28 


52 


29 


15 


22 


lU 


hi 


23 


9 


13 


21 


Sk 


35 


30 


U3 


100 


l8l 


100 


69 


100 



Ul 


111 


53 


h3 


5U 


38 


82 


39 


28 


35 


lU 


12 


6 


6 


8 


7 


3 


1 


2 


3 


100 


208 


99 


79 


100 
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Table 17 reports that 53^ of the ratings for the cassettes were good 
or very good for use with primary grade level children. Eighty-seven 
percent of the ratings indicated that the cassettes were good or very good 
for use with intermediate grade level children. In addition, 2^ of the 
ratings indicated that the tapes as a whole were poor for use with primary 
Children while only four percent of the ratings were reported as being 
poor for intermediate grade children. 

Twenty-one percent of the language arts primary tape ratings «re poor 
compared to only seven percent poor for intermediate level students. 

Social studies primary had good or very good ratings and 1% 
poor ratings compared to the intermediate level of <^ good or very good 
and only one percent poor ratings. 

Mathematics primary grade ratings had kk<i, good or better and kyj, 
poor compared to the intermediate level of 8^ good or very good and 
three percent poor. 

science primary had 59^ good or better and Ik^ poor ratings compared 
to intermediate science ratings of 92^6 good or very good and only three 
percent poor. Thus, at all subject levels, the raters Judged the cassette 
materials to be more suitable for intermediate usage than for primary grade 
level usage. 

Table 18 reports the results for usage according to source of produc- 
tion with primary and intermediate grades. The table indicates that a 
higher percentage of good or very good ratings was accorded to local (57%) 
and commercial (61*) productions than to radio and other sources (2936) 
when the sources were rated for use with primary grades. Ninety-three per- 
cent Of the radio and other sources were given a good or better rating for 
use with intermediate age students; local and commercial productions had 86?^ 
and %% respective ratings indicating good or very good ratings for usage' 
with intermediate children. 35 
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Table l8 



Frequency and Percent Distributions For All Production Sources 
And By Local, Commercial, and Radio or Other Schools When the 
Cassette Tapes Were Judged for Primary or Intermediate 
Grade Level Usage 





All 










Radic 


> and 


Rating 


Productions 


Local 


Commercial 


Other 


N 




N 




N 




N 


I0 


Primary 


















Very Good 


115 




51 


26 


58 


31 


6 


6 


Good 


138 


29 


60 


31 


57 


30 


21 


23 


Fair 


87 


IB 


38 


20 


29 


15 


20 


22 


Poor 


135 


28 


U5 


23 


UU 


23 


U6 


U9 


Total 


U75 


99 


I9I+ 


100 


188 


99 


93 


100 


Intermediate 


















Very Good 


253 


U8 


103 


U8 




U7 


56 


U9 


Good 


206 


39 


80 


38 


75 


38 


51 




Fair 


h9 


9 


19 


9 


2k 


12 


6 


5 


Poor 

Total^ 


20 


k 


11 


5 


7 


U 


2 


2 


528 


100 


213 


100 


200 


101 


115 


100 


*In some instances 


percentage totals do not equal 100^ 


because of 


rounding. 


Table 19 presents 


a breakdown 


by subject 


area 


of the 


number 


of cassette 





tapes that had a good or better rating for usage with primary, intermediate, 
and both primary and intermediate level children. Also reported are those 
tapes which were judged not suitable for either primary or intermediate grade 
levels. The table indicates that more than twice as many cassettes were 
more suitable for intermediate grades than for primary grades. Fifty-one 
of the 1U6 cassettes (35^) were suitable for use with primary grade level 
children and II7 {80%) were suitable for intermediate level children. Thirteen 
or 9$, of the tapes were reported as not being suitable for use with either 
primary or intermediate children (see Appendix B for individual tape grade 
level recommendation) , 
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Table 19^ 



Number and Percent of Cassettes by Subject Area Rated 
Appropriate for Use With Primary, Intermediate and Both 
Primary and Intermediate Grade Levels and Those Tapes 
Rated Not Appropriate for Either Primary or 
Intermediate Grade Level Use 





Inter- 


Primary 


mediate 


N ^ 


N ^ 


13 81 


26 32 


1 6 


Ik 18 


1 6 


2 2 


1 6 


UO U9 


16 99 


82 101 



Both Pri 
mary and 
Intermediate 
N 



Neither Pri- 
mary cr In- 
termediate 
N 



Subject 
Language Arts 
Mathematics 
Science 

Social Studies 



A mean rating by the teacher evaluators of good or verv eood was rpn.Hr.»H 

^ade level use. Where a tape received a mean rating of good or very 
good for use with both primary and intermediate grade levels i? wis 
entered under column k (both primary and intermefiate)! When a Tele 

ZVil"^ °^ eood for both primary anS in^! 

mediate grade use it was placed under column 5. 

^In some instances percentages totals do not equal lOOf, because of rounding. 

Seven of 11 (63^) science tapes were reported not to be suitable for 
K-6 grade level use. However only 13 of the 1U6 tapes (9^6) were reported 
as not being suitable for K-6 use. 
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Conclusions and Recommendations 
The major purposes of this evalmtion were to determine whether 
the general quality and content of the Clinton Cassette Project tapes 
were satisfactory as viewed by ik teacher evaluators. It was also the 
intent of the evaluation to see if the tapes were suitable for use 
with those children qualifying under Title I ESEA guidelines. That is, 
those children who are oi.e or more years below grade level in reading 
or mathematics achievement. 

It should be emphasized that no direct attempt was made to deter- 
mine how efficient or effective the tapes were for instructional purposes 
nor was any attempt made to determine if students that used the tapes 
made any significant improvement in basic skills svibject areas. 

Answers to the following questions were sought in the evaluation: 

1. Which cassettes appeared to be well suited for teaching youngsters 
who are educationally disadvantaged and therefore should be re- 
tained as a part of the library? 

2. Which cassettes needed changes in order to be maintained and 
what kind of changes should be made? 

3. Which cassettes should be discarded because of their limited 
usefulness for teaching educationally disadvantaged youngsters? 

Do the cassettes in a given subject area, such as social studies, 
appear to present more difficulties than cassettes in any of the 
other subject areas such as mathematics or language arts? 

5. Do the cassettes produced by the Clinton Cassette Center measure 
up to cassettes which were produced commercially or by radio and 
other school districts? 

6, Are there any consistent kinds of problems with the casseV;es, 
such as sound quality, length of presentation, content interest, 
etc.? * 

In order to achieve the stated objectives, ik teachers were selected 
to conduct the cassette tape evaluation. One teacher was selected from 
each of the Ik Title I South Pyramid elementary schools by the principal 
of the school where he/she taught. One -hundred fortrsix tapes were evaluated 



by the teachers using a cassette tape rating instrument developed by the 

Research Division of the Minneapolis Public Schools in cooperation with 

the Cassette Center (see Appendix A for the rating form used by the teachers), 

Specific recommendations for objectives one through three are found in 
Appendix B. The following section discusses the remaining three objectives. 

Conclusions 

The general quality of the evaluated tapes was good. Ninety-one per- 
cent Of the ratings and better, depending on the item rated, gave a good 
or very good rating to the foUowing tape quality items: voice clarity, 
pleasantness of voice, sound quality, pacing of materials, length of pre- 
sentation, content relevance, content interest and appropriateness of 
related materials. Little difference was noted among the four subject areas 
Of language arts, social studies, mathematics and science when these areas 
were analyzed on the basis of the above stated tape quality items. All 
subject areas received similar good or very good ratings on the tape quality 
items. Mathematics was the only subject which deviated to any marked degree 
from the other subject areas and it was down only slightly in content interest 
having n% fair and poor ratings. 

The cassette tapes quality was down-graded to any marked extent on 
only one item-amount of "settling time." The overeJJL rating for "settling 
time" was good or better on only 7ki of the ratings. Language arts and 
social studies cassettes were reported in need of improvement in settling 
time. Language arts had 22* poor ratings and social studies 17*. 

When the nine tape quality items were evaluated from a source of 
production standpoint (Cassette Center, commercial or radio and other school 
productions) the foUowing conclusions seem to be warranted: 

^' q2il:irSpesr*''°*^°°' *° ^^8h 
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2. Commercially produced tapes were superior to the other two 
productions on five of the nine tape quality items evaluated. 

3. Cassette Center tape productions were superior to the other 
two production sources on the amount of settling down time 
provided for students at the beginning of the cassette tape. 
Commercial and radio and other school productions were weak 
in this area and are in need of improvement. 

When the cassettes were evaluated for their effect:V/eness in helping 
disabled readers learn the subject matter presented on the various tapes, 88% 
of the ratings judged the tapes to be effective or very effective. Little 
difference was observed among the various subject matter areas for this 
purpose. Science had the best rating with 93% of the ratings being effec- 
tive or very effective. Language arts had the poorest ratings with &H 
of the ratings reporting effective or better ratings. Thus, it appears 
that the tapes can be an effective tool for helping disabled readers 
learn subject matter. 

Radio and other school productions were judged to be inferior to 
both Cassette Center and commercial productions when rated for their 
effectiveness in helping disabled readers learn the subject presented. 
Cassette Center tapes had 91% and commercial productions had 87% effective 
or very effective ratings compared to radio and other school production 
ratings of &2% effective or very effective. 

When the cassette tapes were compared with other instructional 
materials for teaching the subject matter presented on the tapes to disabled 
readers, 70% of the ratings indicated that the cassette tapes were more 
effective than most other kinds of instructional materials on the subject 
covered by the tapes. Radio and other school productions were slightly in- 
ferior to those tapes produced either commercially or by the Cassette Center. 
Language arts and mathematic cassettes received fewer favorable ratings 
when compared with other instructional materials than either social studies 
or science tapes. 

31 



The cassette tapes were also rated for their suitability with poor, 
average, and good readers. Eighty-six percent of the ratings indicated 
that the tapes were suitable for use with poor readers. No differences 
existed among the various subject axeas. Cassette Center produced tapes 
were superior to both of the other production sources when the tapes were 
compared for their use with poor readers. Radio and other school produc- 
tion tapes were slightly inferior to commercially produced tapes. 

The lU cassette tape evaluators also rated the importance of the 
skill, concept or infornation presented for instruction in the tapes. 
Ninety-eight percent of the ratings judged the tapes to be moderately 
important to crucial in importance. Sixty-three percent of the raters' 
responses judged the tapes to be very important or crucial. Individual 
subject areas which received a very important or crucial rating were 
language arts with 62^ favorable responses, social studies 5%» mathe- 
matics Q5i, and science 59j&. The production sources had approximately- 
the same ratings, thus, no major differences were found among the various 
production sources as each did about as well as the other . 

The raters were asked to give an overall recommendation for each 
cassette presentation. They could respond by recommending that the tape 
be kept as is without change, they could recommend that the tape be 
scrapped or they could recommend that the tape be kept if certain changes or 
modifications were made. Seventy percent of the teacher ratings recommended 
that the cassette tapes be kept in their present condition and that no changes 
in the tapes were necessary. Only four percent of the ratings recommended 
that the tapes be scrapped while 26^ of the ratings indicated that certain 
changes should be mde on the tapes. No major differences existed among 

32 
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the various subject areas although mathematics tapes received fewer keep 
ratings than any of the other subject areas which received about the same 
percentage of keep ratings. When production sources were compared Qn the 
keep rating, radio and other school productions were somewhat inferior to 
the other two production sources, i.e., radio and other schools received 65% 
keep ratings compared to Cassette Center production ratings of 72% and 
commercial productions of 71%. 

The cassettes were also evaluated for use with primary (K-3) and 
intermediate (U-6) grade level children. The tapes were rated very good, 
good, fair or poor for use with the two levels. The ratings indicated that 
more than twice as many cassette tapes were more suitable for intermediate 
grades than for primary grade level children. Thirty-five percent of the 
tapes were suitable for use with primary grade levels compared to 80% for 
the intermediate grades. Nine percent of the tapes were reported as not 
being suitable for use with grades K-6. The reader should consult Appendix B 
or C for individual tape grade level recommendations. 

Recommendations 

It appears that the cassette tapes are suitable for use with educa- 
tionally disadvantaged youth and are in keeping with the intent and purpose 
of Title I ESEA guidelines. 

The recommendations which follow are made on the basis of teacher ratings 
which were made during a summer cassette tape evaluation workshop when students 
were not involved. Actual efficiency of the tapes was not directly evaluated. 
Individual tape recommendations are listed in Appendix C and should be con- 
sulted in light of the following suggestions: 

1. Cassette tapes that were recorded from radio broadcasts or 
purchased from other school districts were generally inferior 
to commercial and Cassette Center productions and are in need 
of improvement in the following areas: 
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a. While the overall technical quality is fairly good, 
the amount of student settling down time provided 
at the beginning of the tape is very weak and de- 
finitely needs improvement. 

b. Radio and other school productions were not as good 
as commercial and Cassette Center productions for 
helping disabled readers learn the subject matter 
presented on the tapes. In addition, a greater per- 
centage of modification recommendations was accorded 
to radio and other school productions than either of 
the other two production sources. Therefore, the 
Cassette Center should carefully review these produc- 
tions to see if there is a need for such productions 
since their overall effectiveness received relatively 
low ratings compared to the other two sources of pro- 
duction, 

2. Commercial productions had superior ratings when compared 
with Cassette Center and radio and other school productions 
in their technical quality and were next to Cassette produc- 
tion in their suitability for use with poor readers. However, 
commercially produced tapes definitely need improvement in the 
amount of settling down time provided students at the beginning 
of the tape. 

3. Cassette Center productions had good technical quality, but 
were not as good as those tapes which were produced commer- 
cially. The amount of settling down time provided at the be- 
ginning of the Cassette Center tapes was excellent, and these 
productions had the best ratings for helping poor readers learn 
the subject matter presented. Thus, the only area that might 
be in need of improvement is in the technical ar^a such as 
voice clarity, sound, presentation length, etc, * should be 
emphasized, however, that the cassettes technical qualities 
were given a good rating and the question is one of excellence, 

k. The following recommendations are made with respect to each 
subject area which was evaluated; 

a. Language Arts - Some language arts tapes are in need 

of improvement for their effectiveness in helping disabled 
readers learn the subject matter presented on the tapes. 
In addition, language arts tapes produced by commercial 
or radio and other school districts need improvement in 
the amount of settling down time provided at the beginning 
of the tape, 

b. Social Studies - Some social studies tapes are in need 
of improvement in the amount of settling time provided 

at the beginning of the tape when the tapes were produced 
by commercial or radio and other school districts. 




1 

c. Mathematics - Mathematic tapes were judged to be in need 
i of improvement in making the tapes content more interesting, 

i However, the skills or concepts presented were judged to 

be far superior to tdie other subject areas and only changes 
I that increase the chances of making the tapes more interest- 

j ing should be made. In addition, the clarity of the voice 

making the tapes could be improved. 

{ d. Science - Of the eleven science tapes that were evaluated, 

seven (63%) were judged to be too difficult for K-6 grade 
level children. Future science tapes sho\ild be made less 

, difficult and more suitable for K-6 grade levels. 

5. Since there was not a great deal of difference between Cassette 
Center produced tapes and commercial productions, the cost factor 
for the two productions should be carefully scrutinized to deter- 
mine which is the best value for the money invested in each type 
of production. 

' 6. Moie than twice as many tapes were judged to be suitable for 

use with intermediate grades as for primary grades. Should the 
Center continue producing approximately one primary grade tape 
for every two intermediate grade level tape or should more 
emphasis be placed on producing more primary grade level tapes? 

7. Appendix B lists tapes that are in need of modification. Modifi- 
cation should be made in light of the suggestion presented in 

Appendix B as well as the individual tape ratings presented in Appendix C, 

8. Present and future tapes which are developed should have the 
teacher or child using it evaluate the tape so that any glaring 
errors can be corrected and so that future users can see the 
comments made. These should be filed with the tapes for ready 
reference. 

9. The following suggestions are representative of the comments 
(found in Appendix C) the teacher evaluators made about the 
kinds of technical improvements that should be made in the tapes 
themselves and therefore should serve as a yardstick for buying 
or producing tapes: 

a. The vocabulary which is used in a tape needs explanation 
when it is of a technical nature such as in mathematics 
or science. Don't assume that children understand the 
meaning. 

b. Directions given to students should be made as simple, 
clear and concise as possible. 

c. Children should be instructed to turn off the recorder 
when they are given time to do work on their own. This 
way they don't have to wait if they finish before the 
speaker starts again or they can take more time if needed. 
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When they have completed their work they can turn on the 
tape. Less tape is needed and it does away with the guess 
work of knowing how much time to allow before the speaker 
begins speaking. 

d. Music used in the tapes is a very desirable feature and 
should continue to be used whenever possible. 



e. 



Related materials should be mimeographed, or offset 
rather than dittoed because they are easier to read and 
maintain their permanence better. 

f . When questions are asked about the story read, it is 
oetter to intersperse the story with several thought 
questions rather than having all the questions at the 
end of the tape. 

g. It is always desirable to have motivation questions or 
discussion at the beginning of the tape. 

h. Mathematics tapes should emphasize the why as well as 
the how. 



1. 



Stories with pictures are desirable for poor readers. 

In some instances the tone beU had too much reverbera- 
tion. Perhaps some other signal or cue could be used 
or at least a tone bell with less reverberation. 
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Appendix A 

Cassette Evaluation Form Used by Teacher Bvaluators 
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(1 - 2) 



(11) 



(12) 



(13) 



Minneapolis Public Schools 

Cassette Evaluation by Teachers 
Summer 1971 



Teacher Number (3 . io)Cassette No._ 

Title 



Subject: (check one) 

1 ^Language Arts or Literature 

2 Social Studies 

3 ^Math 

^ Science 

SlJf supplementary materials w4re provided for use with 

this cassette: (check as many as apply) 



1_ 
2_ 

3 



None 



_Books or booklets 



7 



_books in classroom 
or library 

_films trips 



8 



_manipulative materials 



9_ 

0 



_pictures 
_slides 
_study sheets 
_teacher guide 



_transparency 



Did the player operate satisfactorily? 
1 Yes 2 No 



If No, comment: 



(1^) Was this lesson: 

^ appropriate for poor readers than for average or better 

^^^Lr'""*^' ""r?""' "poor" reader is a child who is one 
or more years below grade level) 

^ ^""^ appropriate for average or better than average readers 

tnan for poor readers 

^ Appropriate for good, poor and average readers alike 

^ appropriate for good, poor or average readers 

47 
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«f th?!.T'^*^^ presentation by circling one rating for each 
of the statements listed below (items lU - 25) 



(15) 
(16) 

(17) 
(18) 

(19) 

(20) 
(21) 

(22) 
(23) 

(2U) 
(25) 

(26) 



The voice was clear 

The voice was pleasant for 
children to listen to 

The sound was good 

The "pacing" of the materials 
presented was appropriate for 
the children 

The presentation was of optimum 
length; neither too long nor 
too short 

It provided sufficient "settling 
down" time at the beginning 

The content was relevant for the 
children in my school 

The content was interesting 

The related materials were appro- 
priate for the cassette presenta- 
tion 

It was appropriate for use with 
primary children 

It was appropriate for use with 
intermediate children 



Very 
Good 



Good 
2 

2 
2 



2 

2 

2 
2 

2 
2 
2 



Fair 
3 

3 
3 



3 

3 

3 
3 

3 
3 
3 



Poor 
k 

k 
k 



k 

k 

k 
k 

k 
k 

k 



S effective would this lesson be to help severely 
disabled readers learn the subject presented? severe]^ 

1 Very effective 

2 Effective 

3 Of limited effectiveness 

^ Worthless 
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3 

(27) Compared with other instructional materials you have used, how would 
you rate this cassette for teaching the subject to disabled readers? 

1 Most effective material I know of for this s\ibject 

2 More effective than most materials on this subject 

3 About as effective as most other materials on this subject 

k Less effective than most materials on this subject 

(28) How important is the skill, concept or information presented on this 
cassette for instruction in this subject? 

1 ^Crucial, can't do without it 

2 ^Very important 

3 ^Moderately important 

h ^Not very important 

5 ^Trivial 

(29) Did you use a headset for this presentation? 
1 Yes 2 No 

(30) What is your recommendation for this cassette presentation? 
1 Keep it as is 

2 Scrap it 

3 Keep it, but make the following changes: 



Do you have any other comments? 

Date 

PLEASE DO NOT WRITE BELOW THIS LINE 



Return to: 
Research Division 
Minneapolis Public Schools 
807 N. E. Broadway 

uo ., 40 



(31) Source: l 
(32-33) Use: 
(3^-37) Count 



IMF:dm 
6/18/71 



!" 
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Appendix B 

Cassette Tape Recommendation Listed by Subject 
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Based on teachers ratings, these Cassette tapes are recommended for 
use with educationalOjr disadvantaged children: 



Langiaage Arts 



Primary 

LAOI6 

IAO23 

LA372.05 

IA839 

lA&k3 

IA8U6 

LI500,03 

LI500.OU 

LI500o06 

LI500.IO 

LI50O.U9 
L1500.6U 

Intermediate 



Listening: Sounds We Hear 

Using Our Senses: The Blind Colt 

Aim - Lesson 5 

Big Bill The Duck 

Flapsey Flopper Of The Parinyard 

The Mean Old Elephant 

The Snowy Day 

How To Be A Nature Detective ( 
The Case Of The H\uigry Stranger 
Caps For Sale 
Bread And Jam For Frances 
Curious George Gets A Medal 



LAOI6 Listening: Sounds We Hear 

LAO23 Using Our Senses: The Blind Colt 

LAO3O Distinguishing Between Fact And Opinion: 

Leeuwenhoek And The Little Animals 

LA355 Phonics Short Vowel i 

LA350.02 Vowels Digraphs ie/ei 

LA371.51 The Gift Of The Magi 

LA606 Skills In Gathering Facts 

LA713 Let's Write Letter To A Hits Been 

LA718.00 Let's Write It Backfired 

LA802 Posttest Listening Skills 

LA822 What Are You Doing? 

LA82U Fact And Opinion - The Starry Grotto 

LA8U8 On The Mountains 

LI500.05 h^stery Player At Left End 

LI5OO.25 Slot Car Racing 

LI50O.33 Runaway Slave - Harriet Tubman 

Both Primary and Intermediate 

LAOO9 Visualizing: The Gift 

LAO31 Pondy's Quiet Ride 

LA351 Phonics Short Vowel a 

LA353 Phonics Short Vowel e 

LA801 Pre-Test Listening Skills 

LA803 Auditory Discrimination - 

lASOk Auditory Discrimination - 

U827 Evil Spider 

LA828 The White Snake 

LA832 The Foolish Donkey 

LA8I17 The Greedy Dog 



Sounds 
Rhymes 
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Recommended Tapes (continued) 



Language Arts 

Both Primary and Intermediate (continued ) 

LI5OO.I3 What Mary Jo Shared 

LI5OO.35 Anatole 

LI5OO.38 Benjie 

LI50O.60 The Emperor's New Clothes 

LI50O.6I The Steadfast Tin Soldier 



Social Studies 



Primary 

Intermediate 

SSOUO.03 

SSOUO.ll 

SSOUO.12 

SSO76.52 

SS200.01 

SS200.02 

SS200.03 

SS6OO 

SS6IO 

SS6II 

SS626 

SS63O.I9 

SS65I.O6 

SS775.07 

SS78U.OI 

SS78U.O2 

SS78U.O3 

SS78U.O8 

SS78U.I7 

SS78li.l5 

SS78U.2I 

SS78U.23 

SS78U.2U 

SS8IO.OI 



Paint And Feather And Camouflage 
Florida 
Apollo lU 
Captain Cook 

V/hels Across The Mainland - Plans Are Made 
Wheels Across The Mainland - On The Way 
Wheels Across The Mainland - Chicago 
The Adventures Of Two Great Explorers 
The Life Of The Voyager 
The Great Carrying Place 

The Stagecoach Business In Pioneer Minnesota 

Blackmen And The Fur Trade With The Indians 

Frederick Douglas 

Learning To Use Meips 

Black Elk 

Chief Joseph 

Sitting Bull 

Chief-Hole-In-The-Day 

Young Joseph (Nez Perce) 

Sacagawea 

American Indians - Religions 
American Indians - After Columbus 
The American Indians Growing Up 
Fort Snelling 



Both Primary and Intermediate 

SSOUO.I5 The Boy Who Sailed Around The World 

SS062,62 The First Thanksgiving 

SSIOO.OO Why Dogs And Cats Dislike Each Other 

SS657.O3 Lincoln's Birthday 

SS768 Robert Goes Shopping 

55775.86 The New Home 

55775.87 Indians Of The Southwest - Pueblo Dwellers And An Apache Raid 
SS78U.ll Johnny Cash 

SS78U.20 American Indians - Arts And Cxilture 

. SS8IO.OU The Growth Of Minneapolis 
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Reconunended Tapes (continued) 
Mathemetlcs 

Primary 

MA026.01 Set I Drawing Set 
Intermediate 

MAOI6 Fractions 

MAO63A0 Multiplication 0-9 (U sec.) 100 Basic Facts 

M/V06U.I6 We Subtract Decimals 

MAOO7 Multiplication 0-5 Simplified Succession 

MAOlU Changing The Terms Of Fractions 

MAO33 Division One Digit Divisor 

M/1039 Addition Of Fractions 

MAO57 Subtraction Of Whole Nunibers 

MA063.31 Multiplication 0-9 (l sec.) 100 Basic Facts 

MAO6U.02 We Learn To Write Decimals #1 

Both Primary and Intermediate 

MAO63.27 Addition 0-9 (sec.) 100 Basic Facts 
MA022 The Names Of Parts 

Science 

Primary 

SC813 A Trip To The Autumn Woods 
Intermediate 

SC805 Let's Find Out About The Weather 
SC002 The Thermometer Above Zero 

Both Primary and Intermediate 

SC806 Now We Know (Songs To Learn By) 
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These cassettes, are in need of modification before further use 
with educationally disadvantaged youth. (Appendix C gives suggestions 
for the kind of modification needed.) 

Language Arts 

Primary 

LA6iO Custard the Dragon 
LA835 Cinderella 
LI500.1U In The Forest 

Intermediate 

LA021 Recognizing Cause and Effect: The Upside-Down Animal 

LA02U Visualizing: The Wolf Attack 

IA372.09 Excellent - Lesson 9 

LA708 Let's Write the Mean Song 

LA718.08 Let's Write Fur, Feather, Fin 

LA818 Cause and Effect - The Red Cross 

Both Primary and Intermediate 

JiA302 Learning Phonics Syllables #2 

Social Studies 

Primary 

SS78U,31 Shingibiss 
Intermediate 

SSOUO.OI I Am The Old Explorer Come Travel With Me 

SSOUO.05 Alaska - Hawaii 

55612 Fort Snelling Part I 

55613 Fort Snelling Part II 
SS6lU Fort Snelling Part III 
SS621 Governing A Frontier City 



Tapes to be Modified (continued) 
Social Studies (continued) 

Intermediate (continued) 

55627 Frontier Remedies And Supersitions 

55628 Life On A Pioneer Farm 
SS630.OO William Maricoe and His Balloon 

SS630.IO Changes In Minnesota Manufticturing Since I839 



SS630.18 


The Beginnings of St. Paxa 


SS650.97 


History - Seekers Of The Dream 


SS657.0U 


George Washington - Frontier Colonel 


SS819 


Mexico: Visiting Mexico City 


SS903 


Lief Er ice son 


SS908.01 


The Theft Of Fire 


SS908.02 


Wolf Wisdom 


Both Primary and Intermediate 




None 




Mathematics 


Primary 






None 


Intermediate 




MAOlO 


Ditroduction to Percent 


mohk 


Problem Solving Parts Of A Problem 


MA05I 


How To Read Line Graphs 




390 Basic Facts Difficult Subtraction 


Both Primary and Intermediate 



MA015 

MA021 

mo2k 

MAO3O 



ERIC 



Parts Of A Whole Fraction 
What Is A Fraction? 
Fifths, Sixths, Eighths 
Subtraction Whole Nuniber #1 

k6 



Not Recommended 



These Cassette tapes are not recommended for use with educationally 
disadvantaged youth because they are too difficult or are not suitable 
for use with primary or intermediate children: 



IAOO3 
LAOO8 
IAOI8 
LAO27 
LA366 



Language Arts 

Rashid And The Lion Trainer 
Using Our Sense: Voices And Little Birds 
Recognizing Sequence: The New Boy 
Sharing Feeling: Mora And The Monsoon 
Synonyms Section 2 New Ways In Composition 



SC101.02 

SC8OI 

SC803 

SC807 

SCOOi* 

SCO22.61 

scioi.oi 



Science 

Star Pictures #2 

Let's Find Out What Makes Things 

Let's Find Out About The Moon 

Now We Know (Songs To Learn By) 

Below Zero And The Boiling Point 

Discovering Reptiles 

Star Pictures #1 



Move 



None 



Mathematics 



Social Studies 



SS78»4.32 The Wild White Horse 



ERIC 



h7 



f 1» 



Appendix is available upon reauest onn+f!? !^ V complete 
Evaluation ^part^nt TlXi^^T^l^ ^^^f' 



Appendix C 

Tables Showing Teacher Ratings for 
Individual Tapes by Subject Area 1 



s:ndrAs:n'di"f A^eh^crcLts^Se^^r ^^"-^^ - 

the teachers who eCa^S^^ed ?Se Spef SJiJf 

numbers 15-22 and 26-?ft^«™ ^Pes. Rating instrument item 

headings. correspond with the column criteria 

Grade level designations are listed by letters- ^ « „ 4 /«x 
intermediate (I), both primary and interSeS (^IV ^nJ"^"?*?^' 
primary or intermediati* rwPT\ a •^"^^""e^iate ^Pi; and neither 

wasTe^uired bSSe a ?aif r^t^f" Ti""^ ^^^'^ °' 8°°^ 
and intermedial Jsa^e. 'Shere 1^1^.^!^ ^'^**^^' P^^"^ 
good or very good f or ^saee ^?tJ '^^J*^^®'^ * "«an rating of 
levels it ^s^J^en a PI StSf SSn^'I Jf"^ intermediate grade 

a commercial enterprise (c), or by rLiS^and^tJe^r Strjjts () 

L^::s;;l^o^L"^:c^t"^ra'?^ i^i ^^'^ ^^^^^^-^ 

items 26. 27 and 28 ^ i^.^^i®'" ""^^^ °" ^^^^i'^S form (see 

because Vivace n^J^tSnsifCfs'^"" ^PP^"'^^ A) bit 

exact rating wordagf necessary to leave out the 
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